Extraction of multi-modal online resources associated with research papers

ABSTRACT

A method includes storing a research paper and a set of candidate resources including media content. The method further includes encoding each of one or more first content fields in the research paper into a first vector based on a first field type associated with each of the one or more first content fields. The method further includes encoding each of one or more second content fields in each of the parsed set of candidate resources into a second vector, based on a second field type associated with each of the one or more second content fields. The method further includes comparing the first vector with the second vector to determine a final set of resources based on the comparison. The method further includes controlling a display screen to output the determined final set of resources and the research paper.

FIELD

The embodiments discussed in the present disclosure are related toextraction of multi-modal online resources associated with researchpapers.

BACKGROUND

Typically, a researcher may study a large number of research papers tounderstand latest progress in a domain of interest. For example, aresearcher may read a particular research paper and further identify andstudy one or more relevant research papers cited in the research paper.Generally, the researchers search for the one or more cited researchpapers on a search engine. The study and search may be a time consumingand may be a tedious task based on the domain of interest and priorknowledge of the researcher. In certain solutions, the researcher maymanually search to find online resources (for example video, slides)scattered around web pages to get an overview of the identified researchpapers and the domain of interest. As may be evident, the manual processof search of the online resources associated with the research paper maybe time consuming and cumbersome in case a large number of researchpapers. Thus, there is a need for an enhanced method to search andextract media content of online resources associated with researchpapers for effective understanding of the domain of interest for theresearchers.

The subject matter claimed in the present disclosure is not limited toembodiments that solve any disadvantages or that operate only inenvironments such as those described above. Rather, this background isonly provided to illustrate one example technology area where someembodiments described in the present disclosure may be practiced.

SUMMARY

According to an aspect of an embodiment, a method may be provided. Themethod may comprise extracting, from one or more first websites, one ormore first resources based on a title associated with a research paper.The method may further include identifying a set of resource typesassociated with the extracted one or more first resources. The methodmay further include determining one or more first resource types from apredefined plurality of resource types based on the identified set ofresource types. The set of resource types may exclude the determined oneor more first resource types. The method may further include extracting,from one or more second websites, one or more second resources,associated with the determined one or more first resource types, basedon the title associated with the research paper. The each of the one ormore first resources and the one or more second resources may comprisemedia content. The method may further include determining a final set ofresources based on a comparison between one or more first content fieldsof the research paper and one or more second content fields of theextracted one or more first resources and the extracted one or moresecond resources. The method may further include controlling a displayscreen to output the determined final set of resources and the researchpaper.

According to an aspect of another embodiment, a method may be provided.The method may include storing a set of candidate resources and aresearch paper, where each may include one or more content fields. Theset of candidate resources may comprise media content and may beassociated with the research paper. The method may further includeencoding each of one or more first content fields in the research paperinto a first vector based on a first field type associated with each ofthe one or more first content fields. The method may further includeparsing each of the stored set of candidate resources into one or moresecond content fields. The method may further include encoding each ofthe one or more second content fields in each of the parsed set ofcandidate resources into a second vector, based on a second field typeassociated with each of the one or more second content fields. Themethod may further include comparing the first vector for each of theencoded one or more first content fields with the second vector for eachof the encoded one or more second content fields. The method may furtherinclude determining a final set of resources based on the comparison.The method may further include controlling a display screen to outputthe determined final set of resources and the research paper.

The objects and advantages of the embodiments will be realized andachieved at least by the elements, features, and combinationsparticularly pointed out in the claims.

Both the foregoing general description and the following detaileddescription are given as examples and are explanatory and are notrestrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will be described and explained with additionalspecificity and detail through the use of the accompanying drawings inwhich:

FIG. 1 is a diagram representing an example environment related toextraction of resources associated with a research paper from one ormore websites;

FIG. 2 is a block diagram that illustrates an exemplary electronicdevice for extraction of resources associated with a research paper fromone or more websites;

FIGS. 3A, 3B, and 3C, collectively illustrate, exemplary websites forextraction of resources associated with a research paper;

FIG. 4 illustrates a flowchart of an example method for extraction ofonline resources associated with a research paper;

FIG. 5 illustrates a flowchart of an example method for extraction ofone or more first content fields of a research paper and one or moresecond content fields of each of one or more resources associated withthe research paper;

FIG. 6 illustrates a flowchart of an example method for merger of aplurality of resources associated with a same resource type;

FIG. 7 illustrates a flowchart of an example method for determination ofa final set of resources associated with a research paper;

FIG. 8 illustrates a flowchart of an example method for determination ofa final set of resources associated with a research paper; and

FIG. 9 illustrates an exemplary user interface (UI) that may display afinal set of resources with the research paper,

all according to at least one embodiment described in the presentdisclosure.

DESCRIPTION OF EMBODIMENTS

Some embodiments described in the present disclosure relate to methodsand systems for automated extraction of multi-modal online resourcesincluding media content (for example, videos, posters, presentationsslides, or program codes) associated with research papers. In thepresent disclosure, one or more first resources may be extracted fromone or more first websites based on a title associated with a researchpaper. A set of resource types associated with the one or more extractedfirst resources may be identified. Further, one or more first resourcetypes may be determined from a predefined plurality of resource types(i.e., predefined target resource types to be extracted) based on theidentified set of resource types. The set of resource types of the oneor more extracted first resources may exclude the determined one or morefirst resource types. Herein, the one or more first resource types maycorrespond to those resource types in the predefined plurality ofresource types that may be missing (or excluded) from the set ofresource types associated with the extracted one or more firstresources. One or more second resources associated with the determinedone or more first resource types may be extracted from one or moresecond websites (e.g., a search engine), based on the title associatedwith the research paper. Herein each of the one or more first resourcesand the one or more second resources may include that media content maybe related to the research paper. A final set of resources may bedetermined based on a comparison between one or more first contentfields of the research paper and one or more second content fields ofeach of the extracted one or more first resources and the extracted oneor more second resources. A display screen may be further controlled tooutput the determined final set of resources and the research paper.

In another embodiment described in the present disclosure relates tomethods and systems for automated comparison of extracted multi-modalonline resources (including the media content) with associated researchpapers. In the present disclosure, a set of candidate resources and aresearch paper may be stored. Each of the set of candidate resources andthe research paper may include one or more content fields. The set ofcandidate resources may include the media content and may be associatedwith the research paper. The research paper may be parsed into one ormore first content fields, while each of the stored set of candidateresources may be parsed into one or more second content fields. Each ofthe one or more first content fields in the parsed research paper may beencoded into a first vector based on a first field type associated witheach of the one or more first content fields. Further, each of the oneor more second content fields in the parsed set of candidate resourcesmay be encoded into a second vector based on a second field typeassociated with each of the one or more second content fields. The firstvector for each of the encoded one or more first content fields may becompared with the second vector for each of the encoded one or moresecond content fields. A final set of resources may be determined basedon the comparison and a display screen may be controlled to output thedetermined final set of resources and the research paper.

According to one or more embodiments of the present disclosure, thetechnological field of media content resource extraction may be improvedby configuring a computing system in a manner the computing system maybe able to automatically extract online resources associated with aresearch paper and compare the extracted resources with the associatedresearch paper to output a final set of online resources. The computingsystem may automatically extract the online resources associated withthe research paper from one or more websites as candidate resources andcompare the candidate resources with the research paper to determine afinal set of resources associated with the research paper, as comparedto other conventional solutions which may require significant manualinputs and effort to search and filter the online resources associatedwith the research paper from various websites.

The system may be configured to extract one or more first resources fromone or more first websites, based on a title associated with a researchpaper. For example, the one or more first websites may be a conference,journal, or pre-print research paper publisher website, a personal oracademic website, or a resource-specific website. Herein, a resource mayinclude media content. Examples of the one or more first resources mayinclude, but are not limited to, presentation slides, posters, videos,or program codes. The system may be further configured to identify a setof resource types associated with the extracted one or more firstresources. The system may be further configured to determine one or morefirst resources types from a predefined plurality of resources typesbased on the identified set of resource types. Herein, the set ofresource types may exclude the determined one or more first resourcetypes. For example, the predefined plurality of resource types mayinclude, but is not limited to, a research type, a poster type, apresentation slide type, a video type, or a program code type. In case,the set of resource types includes a research type (i.e., the researchpaper itself), a poster type, and a presentation slide type; then thedetermined one or more first resource types may include, but is notlimited to, a video type or a program code type. In other words, the oneor more first resource types may be missing in the set of resource typesbut present in the plurality of predefined resource types.

The system may be further configured to extract one or more secondresources associated with the determined one or more first resourcetypes from one or more second websites, based on the title associatedwith the resource paper. For example, if the determined one or morefirst resource types include the video type and the program code typeresources, the one or more second websites (e.g., a search engine) maybe searched to extract resources of the video type and the program codetype. The extracted one or more second resources may be different fromextracted the one or more first resources.

The system may be further configured to determine a final set ofresources based on a comparison between one or more first content fieldsof the resource paper and one or more second content fields of theextracted one or more first resources and the extracted one or moresecond resources. For example, a title of the research paper may becompared with a title associated with each of extracted poster,presentation slides, or video (i.e. the extracted one or more firstresources and the extracted one or more second resources). In anexample, an author of the research paper may be compared with an authorassociated with each of the extracted poster, presentation slides, andvideo. Further, text, figures, or tables in the research paper may becompared with corresponding text, figures, and tables in each of theextracted poster, presentation slides, or video. Based on the automaticcomparison and a successful match between the research paper and theextracted resources (e.g., the poster, presentation slides, video, orprogram codes), the matched resources may be determined as the final setof resources. The system may be further configured to control a displayscreen to output the determined final set of resources and the researchpaper, as aggregated multi-modal data. For example, the poster,presentation slides, video, or program codes (i.e., the final set ofresources) may be displayed together with the research paper on anintegrated user interface (UI) for the researcher.

According to one or more other embodiments of the present disclosure,the system may be configured to store a set of candidate resources and aresearch paper, each of which may include one or more content fields.The set of candidate resources may include the media content and the setof candidate resources may be associated with the research paper. Forexample, content related to a poster, presentation slides, a video, or aprogram code may be stored as the set of candidate resources. The systemmay be further configured to parse the research paper into one or morefirst content fields and each of the set of candidate resources into oneor more second content fields. For example, the one or more firstcontent fields may include, but is not limited to, a title, authors,date of publication, abstract, full text, figures, tables, or metainformation associated with research paper. In an example, the one ormore second content fields may include, but is not limited to, a title,authors, date of online upload of the resource, text, figures, tables,or meta information associated with the set of candidate resources.

The system may be configured to encode each of the one or more firstcontent fields in the research paper into a first vector based on afirst field type associated with each of the one or more first contentfields. Similarly, the system may be configured to encode each of theone or more second content fields in the set of candidate resources intoa second vector based on a second field type associated with each of theone or more second content fields. Each of the first field type and thesecond field type may include one of a textual field type, a categoricalfield type, a date-time field type, a figure field type, or a tablefield type.

The system may be configured to compare the first vector for each of theencoded one or more first content fields with the second vector for eachof the encoded one or more second content fields. Based on thecomparison, the system may be configured to determine a final set ofresources from the set of candidate resources. For example, the systemmay add a first candidate resource from the set of candidate resourcesto the final set of resources, in case the first vector for the researchpaper matches with the second vector for the first candidate resource.Based on the comparison, the system may validate that the firstcandidate resource corresponds to the research paper. The system may befurther configured to control a display screen to output the determinedfinal set of resources and the research paper, as the aggregatedmulti-modal data. For example, the final set of resources may bedisplayed together with the research paper on an integrated UI for theresearcher.

Typically, the researcher may wish to obtain an overview of multipleresearch papers of a domain of interest to understand the state of artof the domain. However, in the absence of an automation tool (such asthe disclosed computing system), the researcher may need to manuallysearch and extract online resources related to each research paper. Suchmanual approach may be tedious and time consuming as the researcher mayhave to manually access multiple scattered websites on the Internet andextract or download the accessed resources. The researcher may furtherhave to manually verify whether content in each accessed online resourceactually corresponds to the research paper. As may be evident, themanual process of search of the online resources associated with aresearch paper, and the aggregation of such online resources may be timeconsuming task and may not scale well for a batch of a large number ofresearch papers. In contrast, the disclosed system may provide automatedand enhanced extraction of multi-modal online resources (including mediacontent) associated with a research paper, and provide an automatedvalidation that the extracted online resources correspond to theresearch paper. Such automatic approach may save significant time of theresearcher to search, extract, and/or validate the online resources(i.e. multi modal content) relevant to the research paper.

Embodiments of the present disclosure are explained with reference tothe accompanying drawings.

FIG. 1 is a diagram representing an example environment related toextraction of resources associated with a research paper from one ormore websites, arranged in accordance with at least one embodimentdescribed in the present disclosure. With reference to FIG. 1, there isshown an environment 100. The environment 100 may include an electronicdevice 102, a server 104, a database 106, a user-end device 108, and acommunication network 110. The electronic device 102, the server 104,the database 106, and the user-end device 108 may be communicativelycoupled to each other, via the communication network 110. In FIG. 1,there is further shown one or more first websites 112 and one or moresecond websites 114. The server 104 may host the one or more firstwebsites 112 and the one or more second websites 114. The one or morefirst websites 112 may include a website 112A, a website 112B, . . . anda website 112N. The one or more second websites 114 may include awebsite 114A, a website 114B, . . . and a website 114N. There is furthershown a user 116 who may be associated with or operating the electronicdevice 102 or the user-end device 108.

The electronic device 102 may comprise suitable logic, circuitry,interfaces, and/or code that may be configured to extract one or morefirst resources from the one or more first websites 112 hosted by theserver 104, based on a title associated with a research paper. Forexample, the one or more first websites 112 may include, but are notlimited to, a conference, journal, or pre-print research paper publisherwebsite, a personal or academic website, or a resource-specific website.Herein, a resource may include media content. Examples of the one ormore first resources may include, but are not limited to, presentationslides, posters, videos, or program codes. The electronic device 102 maybe further configured to identify a set of resource types associatedwith the extracted one or more first resources. The electronic device102 may be further configured to determine one or more first resourcestypes from a predefined plurality of resources types based on theidentified set of resource types. Herein, the set of resource types mayexclude the determined one or more first resource types as described,for example, in FIG. 4.

The electronic device 102 may be further configured to extract one ormore second resources associated with the determined one or more firstresource types from the one or more second websites 114 hosted by theserver 104, based on the title associated with the resource paper. In anexample, the one or more second websites 114 may be one or more searchengine websites which may be different from the one or more firstwebsites 112. The electronic device 102 may be further configured todetermine a final set of resources based on a comparison between one ormore first content fields of the resource paper and one or more secondcontent fields of the extracted one or more first resources and theextracted one or more second resources. The one or more first contentfields and the one or more second content fields and the comparison aredescribed, for example, in FIGS. 5 and 7.

Based on the comparison and a successful match between the researchpaper and an extracted resource (e.g., a poster, presentation slides,video, or program code), the electronic device 102 may further determinethe extracted resource as a resource which may be one of the final setof resources. The electronic device 102 may be further configured tocontrol a display screen (such as a display screen 212 shown in FIG. 2)to output the determined final set of resources and the research paper.The extraction and display of the online resources associated with aresearch paper is explained further, for example, in FIGS. 4, 5, 6, and9.

In another embodiment, the electronic device 102 may be configured tostore a set of candidate resources and a research paper, each of whichmay include one or more content fields. The set of candidate resourcesmay include media content and the set of candidate resources may beassociated with the research paper. The electronic device 102 may befurther configured to parse the research paper into one or more firstcontent fields and each of the set of candidate resources into one ormore second content fields as further described, for example, in FIGS. 5and 7.

The electronic device 102 may be further configured to encode each ofthe one or more first content fields in the research paper into a firstvector based on a first field type associated with each of the one ormore first content fields. Similarly, the electronic device 102 may befurther configured to encode each of the one or more second contentfields in the set of candidate resources into a second vector based on asecond field type associated with each of the one or more second contentfields. Each of the first field type and the second field type mayinclude one of a textual field type, a categorical field type, adate-time field type, a figure field type, or a table field type. Theencoding of the one or more first content fields and the one or moresecond content fields is further described, for example, in FIG. 7.

The electronic device 102 may be further configured to compare the firstvector for each of the encoded one or more first content fields with thesecond vector for each of the encoded one or more second content fields.Based on the comparison, the electronic device 102 may be configured todetermine a final set of resources from the set of candidate resources.For example, the electronic device 102 may add a first candidateresource from the set of candidate resources to the final set ofresources, in case the first vector for the research paper matches withthe second vector for the first candidate resource. Based on thecomparison, the electronic device 102 may validate that the firstcandidate resource corresponds to the research paper or not. Theelectronic device 102 may be further configured to control the displayscreen to output the determined final set of resources and the researchpaper. For example, the final set of resources may be displayed togetherwith the research paper (as multi modal content) on an integrated userinterface (UI). The automatic validation of extracted resources with theresearch paper is explained further, for example, in FIGS. 7 and 8. Anexemplary integrated UI that may display the final set of resources andthe research paper is explained further, for example, in FIG. 9.

Examples of the electronic device 102 may include, but are not limitedto, a web wrapper device, a web search device, a search engine, a mobiledevice, a desktop computer, a laptop, a computer work-station, acomputing device, a mainframe machine, a server, such as a cloud server,and a group of servers. In one or more embodiments, the electronicdevice 102 may include a user-end terminal device and a servercommunicatively coupled to the user-end terminal device. The electronicdevice 102 may be implemented using hardware including a processor, amicroprocessor (e.g., to perform or control performance of one or moreoperations), a field-programmable gate array (FPGA), or anapplication-specific integrated circuit (ASIC). In some other instances,the electronic device 102 may be implemented using a combination ofhardware and software.

The server 104 may comprise suitable logic, circuitry, interfaces,and/or code that may be configured to host one or more websites of acategory. For example, the server 104 may host the one or more firstwebsites 112 from which the one or more first resources may be extractedby the electronic device 102. Further, the server 104 may host the oneor more second websites 114 from which the one or more second resourcesmay be extracted by the electronic device 102. Examples of the server104 may include, but are not limited to, a web server, a databaseserver, a file server, a media server, an application server, amainframe server, or a cloud computing server. In one or moreembodiments, the electronic device 102 may include the server 104. Theserver 104 may be implemented using hardware including a processor, amicroprocessor (e.g., to perform or control performance of one or moreoperations), a field-programmable gate array (FPGA), or anapplication-specific integrated circuit (ASIC). In some other instances,the server 104 may be implemented using a combination of hardware andsoftware.

The database 106 may comprise suitable logic, interfaces, and/or codethat may be configured to store the one or more first resources and theone or more second resources that may be extracted by the electronicdevice 102 from the one or more first websites 112 and the one or moresecond websites 114 hosted on the server 104. In an embodiment, thedatabase 106 may store the set of candidate resources and the researchpaper. The database 106 may further store the one or more first contentfields of the research paper and the one or more second content fieldsof each of the one or more first resources and the one or more secondresources (or the one or more second content fields of the set ofcandidate resources). In addition, the database 106 may also store thefinal set of resources that may match with the research paper.

The database 106 may be a relational or a non-relational database. Also,in some cases, the database 106 may be stored on a server, such as acloud server or may be cached and stored on the electronic device 102.In an embodiment, the server of the database 106 may be configured toreceive a request for a research paper, a resource, or a content fieldof a research paper or a resource from the electronic device 102, viathe communication network 110. In response, the server of the database106 may be configured to retrieve and provide the requested researchpaper, resource, or content field to the electronic device 102 based onthe received request, via the communication network 110. Additionally,or alternatively, the database 106 may be implemented using hardwareincluding a processor, a microprocessor (e.g., to perform or controlperformance of one or more operations), a field-programmable gate array(FPGA), or an application-specific integrated circuit (ASIC). In someother instances, the database 106 may be implemented using a combinationof hardware and software.

The user-end device 108 may comprise suitable logic, circuitry,interfaces, and/or code in which the integrated UI including theresearch paper and the final set of resources may be displayed. Theuser-end device 108 may include a web browser software or standalonesoftware to display the integrated UI. In an embodiment, the user-enddevice 108 may receive a user input including the title or a UniformResource Locator (URL) of the research paper from the user 116. Theuser-end device 108 may include a graphical user interface (GUI) toreceive the user input. The user-end device 108 may further provide theuser input to the electronic device 102, via the communication network110, to automatically search and extract the final set of resourcesrelated to the research paper indicated in the user input. The user-enddevice 108 may further receive the final set of resources from theelectronic device 102 based on the provide user input. The web browseror the standalone software may display the integrated UI including theresearch paper and the final set of resources based on the received userinput of the title or the URL of the research paper from the user 116.Examples of the user-end device 108 may include, but are not limited to,a web software development or testing device, a search engine device, amobile device, a desktop computer, a laptop, a computer work-station, acomputing device, a mainframe machine, a server, such as a cloud server,and a group of servers. Although in FIG. 1, the user-end device 108 isseparated from the electronic device 102; however, in some embodiments,the user-end device 108 may be integrated in the electronic device 102,without a deviation from the scope of the disclosure.

The communication network 110 may include a communication medium throughwhich the electronic device 102 may communicate with the server 104, theserver which may store the database 106, and the user-end device 108.Examples of the communication network 110 may include, but are notlimited to, the Internet, a cloud network, a Wireless Fidelity (Wi-Fi)network, a Personal Area Network (PAN), a Local Area Network (LAN),and/or a Metropolitan Area Network (MAN). Various devices in theenvironment 100 may be configured to connect to the communicationnetwork 110, in accordance with various wired and wireless communicationprotocols. Examples of such wired and wireless communication protocolsmay include, but are not limited to, at least one of a TransmissionControl Protocol and Internet Protocol (TCP/IP), User Datagram Protocol(UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP),ZigBee, EDGE, IEEE 802.11, light fidelity(Li-Fi), 802.16, IEEE 802.11s,IEEE 802.11g, multi-hop communication, wireless access point (AP),device to device communication, cellular communication protocols, and/orBluetooth (BT) communication protocols, or a combination thereof.

Modifications, additions, or omissions may be made to FIG. 1 withoutdeparting from the scope of the present disclosure. For example, theenvironment 100 may include more or fewer elements than thoseillustrated and described in the present disclosure. For instance, insome embodiments, the environment 100 may include the electronic device102 but not the database 106 and the user-end device 108. In addition,in some embodiments, the functionality of each of the database 106 andthe user-end device 108 may be incorporated into the electronic device102, without a deviation from the scope of the disclosure.

FIG. 2 is a block diagram that illustrates an exemplary electronicdevice for extraction of resources associated with a research paper fromone or more websites, arranged in accordance with at least oneembodiment described in the present disclosure. FIG. 2 is explained inconjunction with elements from FIG. 1. With reference to FIG. 2, thereis shown a block diagram 200 of a system 202 including the electronicdevice 102. The electronic device 102 may include a processor 204, amemory 206, a persistent data storage 208, an input/output (I/O) device210, a display screen 212, and a network interface 214.

The processor 204 may comprise suitable logic, circuitry, and/orinterfaces that may be configured to execute program instructionsassociated with different operations to be executed by the electronicdevice 102. For example, some of the operations may include extractingthe one or more first resources, identifying the set of resource types,determining the one or more first resource types, and extracting the oneor more second resources. The operations may further include storing theset of candidate resources and the research paper, parsing the researchpaper and each of the stored set of candidate resources, encoding eachof the one or more first content fields, encoding each of the one ormore second content fields, and comparing the first vector with thesecond vector. The operations may further include determining the finalset of resources and controlling the display screen (e.g., the displayscreen 212) to display the final set of resources. The processor 204 mayinclude any suitable special-purpose or general-purpose computer,computing entity, or processing device including various computerhardware or software modules and may be configured to executeinstructions stored on any applicable computer-readable storage media.For example, the processor 204 may include a microprocessor, amicrocontroller, a digital signal processor (DSP), anapplication-specific integrated circuit (ASIC), a Field-ProgrammableGate Array (FPGA), or any other digital or analog circuitry configuredto interpret and/or to execute program instructions and/or to processdata.

Although illustrated as a single processor in FIG. 2, the processor 204may include any number of processors configured to, individually orcollectively, perform or direct performance of any number of operationsof the electronic device 102, as described in the present disclosure.Additionally, one or more of the processors may be present on one ormore different electronic devices, such as different servers. In someembodiments, the processor 204 may be configured to interpret and/orexecute program instructions and/or process data stored in the memory206 and/or the persistent data storage 208. In some embodiments, theprocessor 204 may fetch program instructions from the persistent datastorage 208 and load the program instructions in the memory 206. Afterthe program instructions are loaded into the memory 206, the processor204 may execute the program instructions. Some of the examples of theprocessor 204 may be a GPU, a CPU, a RISC processor, an ASIC processor,a CISC processor, a co-processor, and/or a combination thereof.

The memory 206 may comprise suitable logic, circuitry, interfaces,and/or code that may be configured to store program instructionsexecutable by the processor 204. In certain embodiments, the memory 206may be configured to store operating systems and associatedapplication-specific information. The memory 206 may includecomputer-readable storage media for carrying or havingcomputer-executable instructions or data structures stored thereon. Suchcomputer-readable storage media may include any available media that maybe accessed by a general-purpose or special-purpose computer, such asthe processor 204. By way of example, and not limitation, suchcomputer-readable storage media may include tangible or non-transitorycomputer-readable storage media including Random Access Memory (RAM),Read-Only Memory (ROM), Electrically Erasable Programmable Read-OnlyMemory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other opticaldisk storage, magnetic disk storage or other magnetic storage devices,flash memory devices (e.g., solid state memory devices), or any otherstorage medium which may be used to carry or store particular programcode in the form of computer-executable instructions or data structuresand which may be accessed by a general-purpose or special-purposecomputer. Combinations of the above may also be included within thescope of computer-readable storage media. Computer-executableinstructions may include, for example, instructions and data configuredto cause the processor 204 to perform a certain operation or group ofoperations associated with the electronic device 102.

The persistent data storage 208 may comprise suitable logic, circuitry,interfaces, and/or code that may be configured to store programinstructions executable by the processor 204, operating systems, and/orapplication-specific information, such as logs and application-specificdatabases. The persistent data storage 208 may include computer-readablestorage media for carrying or having computer-executable instructions ordata structures stored thereon. Such computer-readable storage media mayinclude any available media that may be accessed by a general-purpose ora special-purpose computer, such as the processor 204.

By way of example, and not limitation, such computer-readable storagemedia may include tangible or non-transitory computer-readable storagemedia including Compact Disc Read-Only Memory (CD-ROM) or other opticaldisk storage, magnetic disk storage or other magnetic storage devices(e.g., Hard-Disk Drive (HDD)), flash memory devices (e.g., Solid StateDrive (SSD), Secure Digital (SD) card, other solid state memorydevices), or any other storage medium which may be used to carry orstore particular program code in the form of computer-executableinstructions or data structures and which may be accessed by ageneral-purpose or special-purpose computer. Combinations of the abovemay also be included within the scope of computer-readable storagemedia. Computer-executable instructions may include, for example,instructions and data configured to cause the processor 204 to perform acertain operation or group of operations associated with the electronicdevice 102.

In some embodiments, either of the memory 206, the persistent datastorage 208, or combination may store the one or more first resource,the one or more second resources, the set of candidate resources, thefinal set of resources, and the research papers. Either of the memory206, the persistent data storage 208, or combination may further storethe one or more first content fields of the research paper and the oneor more second content fields of each of the one or more first resourcesand the one or more second resources (and/or one or more second contentfields of the set of candidate resources).

The I/O device 210 may include suitable logic, circuitry, interfaces,and/or code that may be configured to receive a user input. For example,the I/O device 210 may receive the user input to including the title ofthe research paper or the URL of the research paper. The I/O device 210may be further configured to provide an output in response to the userinput. For example, the output may include the integrated UI that maydisplay the final set of resources and the research paper. The I/Odevice 210 may include various input and output devices, which may beconfigured to communicate with the processor 204 and other components,such as the network interface 214. Examples of the input devices mayinclude, but are not limited to, a touch screen, a keyboard, a mouse, ajoystick, and/or a microphone. Examples of the output devices mayinclude, but are not limited to, a display and a speaker.

The display screen 212 may comprise suitable logic, circuitry,interfaces, and/or code that may be configured to render the integratedUI that may display the final set of resources and the research paper.The display screen 212 may be configured to receive the user input fromthe user 116. The user input may include the title or the URL of theresearch paper. In such cases the display screen 212 may be a touchscreen to receive the user input. The display screen 212 may be realizedthrough several known technologies such as, but not limited to, a LiquidCrystal Display (LCD) display, a Light Emitting Diode (LED) display, aplasma display, and/or an Organic LED (OLED) display technology, and/orother display technologies.

The network interface 214 may comprise suitable logic, circuitry,interfaces, and/or code that may be configured to establish acommunication between the electronic device 102, the server 104, thedatabase 106, and the user-end device 108, via the communication network110. The network interface 214 may be implemented by use of variousknown technologies to support wired or wireless communication of theelectronic device 102 via the communication network 110. The networkinterface 214 may include, but is not limited to, an antenna, a radiofrequency (RF) transceiver, one or more amplifiers, a tuner, one or moreoscillators, a digital signal processor, a coder-decoder (CODEC)chipset, a subscriber identity module (SIM) card, and/or a local buffer.

Modifications, additions, or omissions may be made to the exampleelectronic device 102 without departing from the scope of the presentdisclosure. For example, in some embodiments, the example electronicdevice 102 may include any number of other components that may not beexplicitly illustrated or described for the sake of brevity.

FIGS. 3A, 3B, and 3C, collectively illustrate, exemplary websites forextraction of resources associated with a research paper, arranged inaccordance with at least one embodiment described in the presentdisclosure. FIGS. 3A, 3B, and 3C are explained in conjunction withelements from FIG. 1 and FIG. 2.

With reference to FIG. 3A, there is shown a first website 300A. Thefirst website 300A may be a website associated with a conference,journal, or pre-print research paper publisher, which may be associatedwith or store research papers and one or more resources associated witheach research paper. In FIG. 3A, there is shown, a name (such as,“Conference website”, denoted by 302) of the first website 300A. Thefirst website 300A may include a search box (such as, a search box 304A)to search for research papers by metadata and/or content fieldsassociated with the research papers. For example, the electronic device102 may receive a user input through the search box 304A from the user116. The user input may include a title associated with a researchpaper. The first website 300A may search for the research paper based onthe title and present information related to the searched researchpaper.

As shown in FIG. 3A, the first website 300A may display a titleassociated with the searched research paper (such as, “Research PaperTitle: Title-1” denoted by 304B). The first website 300A may alsodisplay links of related resources associated with the searched researchpaper. For example, the first website 300A may display a link of aposter (e.g., Related Resources Links: “Poster-1”, denoted by 304C)associated with the searched research paper. The first website 300A mayfurther display name(s) of author(s) of the searched research paper(such as, “Authors: FirstName A LastName A, FirstName B LastName B,FirstName C LastName C, and FirstName D LastName D” denoted by 304D).The first website 300A may further display an abstract of the searchedresearch paper (such as, “Abstract Text-1” denoted by 304E). Further,the first website 300A may display a full text of the searched researchpaper or a link to a file (such as, “Full Text Links: HTML-1 | PDF-1”denoted by 304F) including the full text. For example, the first website300A may display a link to a Hypertext Markup Language (HTML) file(e.g., “HTML-1”) of the full text or a link to a Portable DocumentFormat (PDF) file (e.g., “PDF-1”) of the full text. The first website300A may further display bibliographic data (such as, denoted by 306) ofthe searched research paper. Examples of the bibliographic data mayinclude, a volume of conference proceedings (e.g., “Conference C-1,Volume 1”), a date of publication (e.g., “April 2020”), page numbers(e.g., “25-28”), and author information (e.g., “FirstName A LastName Aand FirstName B LastName B from ABC University, US; and FirstName CLastName C and FirstName D LastName D from XYZ Inc., US”) as shown inFIG. 3A.

With reference to FIG. 3B, there is shown a second website 300B. Thesecond website 300B may be a website associated with an academic or apersonal webpage of an author of a research paper and/or an author ofone or more resources associated with the research paper. There is shownin FIG. 3B, a name (such as, “Personal website” denoted by 308) of thesecond website 300B. The second website 300B may include an author name(such as, “Author name: FirstName A LastName A”, denoted by 310). Thesecond website 300B may further include a photo or portrait picture ofthe author (such as, a photo 312). As shown in FIG. 3B, the secondwebsite 300B may further include an introduction of the author includinghis/her profession, designation, organization, affiliation, or academicor personal interests. For example, the introduction of the author maybe “I am a Professor of Computer Science at ABC University, US. Myresearch interests include Artificial Intelligence, Cloud Computing,IoT, and Image Processing”, as denoted by 314 in FIG. 3B.

As shown in FIG. 3B, the second website 300B may further include a listof publications (such as, denoted by 316) of the author (e.g.,“FirstName A LastName A”). For example, the list of publications (suchas, denoted by 316) may include a first research paper (such as,“Research Paper Title: Title-1” denoted by 316A) and a second researchpaper (such as, “Research Paper Title: Title-2” denoted by 316B). Thefirst research paper may have a title as “Title-1”, authors as“FirstName A LastName A, FirstName B LastName B, FirstName C LastName C,and FirstName D LastName D”, and an abstract as “Abstract Text-1”. Thesecond website 300B may display a link to an HTML file (e.g., “HTML-1”)and a link to a PDF file (e.g., “PDF-2”). The second website 300B mayfurther display one or more resources related to the first researchpaper. For example, the second website 300B may display a link of aposter (e.g., “Poster-1”) and a link of presentation slides (e.g.,“Presentation Slides-1”) as the one or more resources associated withthe first research paper.

The second research paper (such as, denoted by 316B) may have a title as“Title-2”, authors as “FirstName E LastName E, FirstName A LastName A,FirstName B LastName B, and FirstName C LastName C”, and an abstract as“Abstract Text-2”. The second website 300B may display a link to an HTMLfile (e.g., “HTML-2”) and a link to a PDF file (e.g., “PDF-3”). Thesecond website 300B may further include and display one or moreresources related to the second research paper. For example, the secondwebsite 300B may display a link of a program code (e.g., “SourceCode-2”) and a link of presentation slides (e.g., “PresentationSlides-2”) as one or more resources associated with the second researchpaper.

With reference to FIG. 3C, there is shown a third website 300C. Thethird website 300C may be a website associated with specific resources(such as, posters, presentation slides, videos, program codes or othermedia content) associated with research papers. The third website 300Cmay include databases to store specific resources which may be relatedto the research papers which may be searched by the user 116 (such asresearcher). In an embodiment, the third website 300C may store one typeof resources (for example videos). In another embodiment, the thirdwebsite 300C may store multiple types of resources (for examplecombination of posters, presentation slides, videos, or program codes).

For example, there is shown in FIG. 3C, a name (such as, “Media contentwebsite” denoted by 318) of the third website 300C. The third website300C may include a search box (such as, a search box 320A) to search forresources including the media content associated with the researchpaper. For example, the electronic device 102 may receive the user inputthrough the search box 320A from the user 116. The user input mayinclude a title associated with the research paper. The third website300C may search for the research paper based on the title and presentinformation related to the searched research paper, and further searchfor resources available on the third website 300C for the searchedresearch paper.

As shown in FIG. 3C, as a result of search, the third website 300C maydisplay a title associated with the searched research paper (such as,“Title-1” denoted by 320B). The third website 300C may further displayname(s) of author(s) of the searched research paper (such as, “Authors:FirstName A LastName A, FirstName B LastName B, FirstName C LastName C,and FirstName D LastName D” as denoted by 320C in FIG. 3B). The thirdwebsite 300C may further display bibliographic data (such as, denoted by320D) of the searched research paper. Examples of the bibliographic datamay include, a volume of conference proceedings (e.g., “Conference C-1,Volume 1”), a date of publication (e.g., “April 2020”), page numbers(e.g., “25-28”), and author information (e.g., “FirstName A LastName Aand FirstName B LastName B from ABC University, US; and FirstName CLastName C and FirstName D LastName D from XYZ Inc., US”) as shown inFIG. 3C.

The third website 300C may further display links of resources associatedwith the searched research paper. For example, the third website 300Cmay display links (such as, denoted by 322) to view and/or downloadmedia content (e.g., “Media Content-1”). The media content may includeresources associated with the searched research paper. Examples of themedia content may include, but are not limited to, posters, presentationslides, videos, and/or program codes. The poster and the presentationslides, as the resource, may provide quick and informative summary ofthe research paper. The video, as the resource, may provide vividinformation for better understanding of the domain or concept includedin the searched research paper. The program codes, as the resource, mayprovide information about detailed implementation related to theconcepts mentioned in the searched research paper.

It should be noted that the first website 300A, the second website 300B,and the third website 300C shown in FIGS. 3A-3C are presented merely asexamples and should not be construed to limit the scope of thedisclosure.

FIG. 4 illustrates a flowchart of an example method for extraction ofonline resources associated with a research paper, arranged inaccordance with at least one embodiment described in the presentdisclosure. FIG. 4 is explained in conjunction with elements from FIG.1, FIG. 2, FIG. 3A, FIG. 3B, and FIG. 3C. With reference to FIG. 4,there is shown a flowchart 400. The method illustrated in the flowchart400 may start at 402 and may be performed by any suitable system,apparatus, or device, such as by the example electronic device 102 ofFIG. 1 or FIG. 2. Although illustrated with discrete blocks, the stepsand operations associated with one or more of the blocks of theflowchart 400 may be divided into additional blocks, combined into fewerblocks, or eliminated, depending on the particular implementation.

At block 402, one or more first resources may be extracted from one ormore first websites 112. In an embodiment, the processor 204 may beconfigured to extract the one or more first resources from the one ormore first websites 112 based on a title associated with a researchpaper. Examples of the one or more first websites 112 may include, butare not limited to, a conference, journal, or pre-print research paperpublisher website (e.g., the first website 300A), a personal or academicwebsite (e.g., the second website 300B), or a resource-specific website(e.g., the third website 300C). In an embodiment, the processor 204 mayextract a first set of resources from the first website 300A, a secondset of resources from the second website 300B, and a third set ofresources from the third website 300C. The one or more first resourcesmay include the first set of resources, the second set of resources, andthe third set of resources extracted from the one or more first websites112. Examples of a resource in the one or more first resources mayinclude, but are not limited to, presentation slides, posters, videos,or program codes. In an embodiment, the processor 204 may also extractthe research paper from the one or more first websites 112. For example,along with the first set of resources, the processor 204 may extract theresearch paper from the first website 300A. In an embodiment, theprocessor 204 may extract the one or more first resources from the oneor more first websites 112 based on the title of the research paper. Forexample, the processor 204 may provide the title of the research paper(which may be of interest to the user 116 or any other researcher orknowledge worker) to the first website 300A, the second website 300B, orthe third website 300C to extract the one or more first resources (likeposter, presentation slides, video, or program codes) which may alsoinclude or indexed by the same title (i.e. similar to the title of theresearch paper). In some embodiments, the processor 204 may extract theresearch paper from the one or more first websites 112 based on thetitle (or URL) of the research paper. In other embodiments, theprocessor 204 may be configured to extract the one or more firstresources from the one or more first websites 112 based on other metainformation of the research paper, such as, but not limited to, anauthor name, one or more keywords of abstract or description of theresearch paper.

In some embodiments, the processor 204 may be configured to crawl afirst set of web pages associated with the first website 300A (e.g., awebsite associated with a conference, a journal, or a pre-print researchpaper publisher) for extraction of information (e.g., the first set ofresources) from the first set of web pages. Further, the processor 204may label or select one or more items in a sample web page of the firstset of crawled web pages, based on a user input from a user (e.g., theuser 116). The processor 204 may generate an initial extraction rule forextraction of a first item from the labelled one or more items of thesample web page based on a tree data structure and visual informationassociated with the first item. The processor 204 may refine the initialextraction rule to generate a new extraction rule for extraction of asecond item (i.e. corresponding to the first item) from a target webpage in the first set of crawled web pages. The processor 204 mayautomatically and incrementally refine the new extraction rule for othertarget web pages for extraction of an item from each such target webpage based on the visual information associated with the item to beextracted. The processor 204 may further extract the item from each suchtarget web page in the first website 300A. The extracted item maycorrespond to the research paper and the first set of resources in theone or more first resources.

In some embodiments, similar to the extraction of the information (e.g.,the first set of resources) from the first set of web pages associatedwith the first website 300A, the processor 204 may extract informationfrom a second set of web pages associated with the third website 300C(e.g., a resource-specific website). The information extracted from thethird website 300C may correspond to the third set of resources in theone or more first resources. In an example, in case the third website300C hosts video resources associated with research papers, the thirdset of resources may include a video associated with the research paper.

In some embodiments, the processor 204 may be configured toautomatically identify and extract distributed online resources (e.g.,the second set of resources) from the second website 300B. The processor204 may be configured to locate a candidate entry list page in thesecond website 300B. For example, the candidate entry list page may belocated in the second website 300B based on a keyword search for atarget keyword in links in a home page of the second website 300B.Herein, each link may include anchor text and/or a URL. A link list oflinks that include the target keyword may be created and a link from thelink list may be selected as pointing to the candidate entry list page.The processor 204 may be configured to verify the candidate entry listpage as an entry list page using repeated pattern discovery through aDocument Object Model (DOM) tree of the candidate entry list page. Theprocessor 204 may segment the entry list page into a plurality of entryitems. From the plurality of entry items, the processor 204 may extracta plurality of candidate target pages by identification of one or morelinks in each entry item. Each of the one or more links may point to acorresponding one of the plurality of candidate target pages. Theprocessor 204 may be configured to verify at least one of the candidatetarget pages as a target page based on analysis of a visual structureand presentation of the at least one of the candidate target pages toidentify one or more information blocks in the at least one of thecandidate target pages. The processor 204 may extract one or morekeyword features from the one or more information blocks. The processor204 may further classify the at least one candidate target pages ascorresponding to a certain genre of pages and thereby as the target pagebased on at least one of the one or more keyword features. Finally, theprocessor 204 may then extract metadata and information from the targetpage, as the second set of resources extracted from the second website300B (i.e. the website associated with an academic or a personalwebpage).

For example, FLA11-025 U.S. Pat. No. 9,390,166 B2 filed on Dec. 31,2012, which is incorporated by reference herein in its entirety,discusses online resource identification and extraction in detail. Itmay be noted that methods to extract information from web pages by thereferenced application are merely an example. Although, there may bedifferent other ways to extract information from web pages, withoutdeparture from the scope of the disclosure.

In an embodiment, the processor 204 may be further configured to indexthe extracted items corresponding to the research paper and the one ormore first resources, and store the indexed extracted items in thedatabase 106, the memory 206, the persistent data storage 208, or acombination thereof. In another embodiment, the processor 204 may beconfigured to separately index items corresponding to the research paperand/or the first set of resources, items corresponding to the second setof resources, and items corresponding to the third set of resources. Insuch scenario, the processor 204 may store such separately indexed itemsin separate databases, such as a first database (for the research paperand/or the first set of resources), a second database (for the secondset of resources), and a third database (for the third set ofresources). In an embodiment, the first database, the second database,and the third database may be associated with the database 106, thememory 206, the persistent data storage 208, or a combination thereof.

In an example, the first database may correspond to a database forcentral domain-specific data which may include the research paper andthe first set of resources extracted from the first website 300A (e.g.,a conference, journal, or pre-print research paper publication website).The second database may correspond to a database for scattereddomain-specific data which may include the second set of resourcesextracted from the second website 300B (e.g., an academic or personalwebsite). The third database may correspond to a database for specificresources data which may include the third set of resources extractedfrom the third website 300C (e.g., a resource-specific website).

At block 404, a set of resource types associated with the extracted oneor more first resources may be identified. In an embodiment, theprocessor 204 may be configured to identify the set of resource typesassociated with the extracted one or more first resources extracted fromthe one or more first websites 112. In some embodiment, the processor204 may identify the resources type of a resource (i.e. in the extractedone or more first resources) as a video type or a program code typebased on a URL associated with the resource. For example, the processor204 may compare the URL of the resource with a list of known URLs ofresources of video or program code type to determine whether theresource is of the video type or the program code type respectively.

In some embodiments, the processor 204 may further identify the resourcetype of the resource, such as the research paper, presentation slides,or posters, based on page layout analysis. The processor 204 mayidentify the resource type (i.e. in the set of resource types) of suchresources as the research paper type, the presentation slide type, orthe poster type. The processor 204 may convert each page in suchresources into an image. For example, the processor 204 may convert thepage into a corresponding image based on screen capture ordocument-to-image conversion technique. The processor 204 may identifythe resource type of the resources by application of a pre-trained imageclassifier on the image. The image classifier may be pre-trained usingdeep learning techniques to classify the extracted resources (orconverted images) into the corresponding resource types as one of theresearch paper type, the presentation slide type, or the poster typebased on a layout of the page associated with the corresponding image.

At block 406, one or more first resource types may be determined from aplurality of predefined resource types based on the identified set ofresource types. In an embodiment, the processor 204 may be configured todetermine the one or more first resource types from the plurality ofpredefined resource types based on the set of resource types identifiedfrom the extracted one or more first resources. In an embodiment, theset of resource types may exclude the one or more first resource types.For example, the predefined plurality of resource types may include, butis not limited to, a research paper type, a poster type, a presentationslide type, a video type, or a program code type. In case, the set ofresource types includes a research paper type (i.e., the research paperitself), the poster type, and the presentation slide type; then thedetermined one or more first resource types may include the video typeand the program code type. In other words, the one or more firstresource types may be missing in the set of resource types but presentin the plurality of predefined resource types. In other words, theprocessor 204 may determine the missing types of resources (i.e. one ormore first resource types) to determine the types of resources (forexample the video type or the program code type) which may not beextracted from the one or more first websites 112 at step 402 or notavailable in the one or more first websites 112.

At block 408, one or more second resources associated with the one ormore first resource types may be extracted from one or more secondwebsites 114 based on the title associated with the research paper. Inan embodiment, the processor 204 may be configured to extract the one ormore second resources associated with the one or more first resourcetypes (determined at step 406) from the one or more second websites 114,based on the title associated with the research paper. The server 104may host the one or more second websites 114. Examples of the one ormore second websites may include, but are not limited to, one or moresearch engine websites. In an embodiment, each of the one or more firstresources (extracted at step 402) and the one or more second resources(extracted at step 406) may include the media content for examples, butnot limited to, presentation slides, posters, videos, or program codes.In an example, if the one or more first resources types (determined atstep 406) includes the video type and the program code type, the one ormore second resources may include the video and the program code (i.e.resources that were missing in the one or more first resources extractedfrom the one or more first websites 112 at step 402).

In an example, the processor 204 may extract or search for the resourcesof the video resource type on the search engines associated with videocontent, based on the title of the research paper. For search orextraction of the resources of the program code type, the processor 204may search on the search engines or databases including public orprivate source-code repositories. In an example, in case of thedetermined one or more first resources types as the presentation slidetype or the poster type, the processor 204 may search for thepresentation slides and the posters on generic search engines based onthe title of the research paper and one or more search operators (forexample a filetype restriction, such as, “filetype: pdf” for PortableDocument Format (PDF) files or “filetype: ppt” for presentation slides).

At block 410, a final set of resources may be determined based on acomparison between one or more first content fields of the researchpaper and one or more second content fields of the extracted one or morefirst resources and the one or more second resources. In an embodiment,the processor 204 may be configured to determine the final set ofresources based on the comparison between the one or more first contentfields of the research paper and the one or more second content fieldsof the extracted one or more first resources and the extracted one ormore second resources (i.e. extracted at steps 402 and 408). Theextraction of the one or more first content fields and the one or moresecond content fields is explained further, for example, in FIG. 5. Insome embodiments, the final set of resources may be determined based onmerger of the extracted one or more first resources and extracted theone or more second resources of same type as described further, forexample, in FIG. 6. The determination of the final set of resources isdescribed further, for example, in FIG. 7 and FIG. 8.

At block 412, a display screen may be controlled to output the final setof resources and the research paper. In an embodiment, the processor 204may be configured to control the display screen (such as, the displayscreen 212 of the electronic device 102) to output the determined finalset of resources and the research paper. In some embodiments, theprocessor 204 may output an integrated UI that may display the final setof resources and the research paper together on the display screen 212.An exemplary integrated UI that may display the final set of resourcesand the research paper is described, for example, in FIG. 9. Control maypass to end.

Although the flowchart 400 is illustrated as discrete operations, suchas 402, 404, 406, 408, 410, and 412. However, in certain embodiments,such discrete operations may be further divided into additionaloperations, combined into fewer operations, or eliminated, depending onthe particular implementation without detracting from the essence of thedisclosed embodiments.

FIG. 5 illustrates a flowchart of an example method for extraction ofone or more first content fields of a research paper and one or moresecond content fields of each of one or more resources associated withthe research paper, arranged in accordance with at least one embodimentdescribed in the present disclosure. FIG. 5 is explained in conjunctionwith elements from FIG. 1, FIG. 2, FIG. 3A, FIG. 3B, FIG. 3C, and FIG.4. With reference to FIG. 5, there is shown a flowchart 500. The methodillustrated in the flowchart 500 may start at 502 and may be performedby any suitable system, apparatus, or device, such as by the exampleelectronic device 102 of FIG. 1 or FIG. 2. Although illustrated withdiscrete blocks, the steps and operations associated with one or more ofthe blocks of the flowchart 500 may be divided into additional blocks,combined into fewer blocks, or eliminated, depending on the particularimplementation.

At block 502, one or more first meta fields may be extracted from theresearch paper and one or more second meta fields may be extracted fromeach of the one or more first resources and one or more secondresources. In an embodiment, the processor 204 may be configured toextract the one or more first meta fields from the research paper andthe one or more second meta fields from at least one resource of the oneor more first resources and the one or more second resources (extractedin steps 402 and 408 in FIG. 4). In certain embodiments, the at leastone resource may include each of the one or more first resources and theone or more second resources. In another embodiment, the at least oneresource may include the one or more first resources (or just the firstset of resources extracted from the first website 300A).

In some embodiments, the processor 204 may be configured to search thedatabase 106 based on the title of the research paper to determine a setof meta fields associated with the research paper. In an example, theprocessor 204 may search the first database (i.e. the database forcentral domain-specific data) of the database 106 based on the title ofthe research paper to determine the set of meta fields associated withthe research paper. The first database may include the research paperand the first set of resources that may be extracted from the firstwebsite 300A. As the first website 300A may include more comprehensivemeta fields of the research paper, the processor 204 may check whetherall comprehensive meta fields in the set of meta fields have alreadybeen extracted from the research paper, the one or more first resources,and/or the one or more second resources. In an embodiment, in case allmeta fields in the set of meta fields are not extracted for the researchpaper, the one or more first resources, and/or the one or more secondresources, the processor 204 may perform step 502 to extract the one ormore first meta fields and the one or more second meta fields.

Examples of the one or more first meta fields of the research paper mayinclude, but is not limited to, a title, author(s), institute(s), venueof a conference or journal, a date of paper presentation, submission, orthe conference. In an embodiment, the processor 204 may determine theone or more first meta fields based on layout analysis of a content fileassociated with the research paper. For example, the processor 204 mayuse a PDF extraction technique for extraction of the one or more firstmeta fields from a PDF file of the research paper. In another example,the processor 204 may use an Optical Character Recognition (OCR)technique for extraction of the one or more first meta fields from ascanned document of the research paper.

In an embodiment, similar to the extraction of the one or more firstmeta fields, the processor 204 may extract the one or more second metafields from the extracted resources (such as video, posters,presentation slides, or program codes) associated with the researchpaper. The processor 204 may be configured to extract the one or moresecond meta fields of a video by extraction of a title, a description,author(s), and a date-time of a video post. The processor 204 may beconfigured to extract the one or more second meta fields of a programcode by extraction of author(s) name, a description, a readme text, anda date-time of an upload of the program code.

At block 504, the one or more first content fields and the one or moresecond content fields may be extracted based on text extraction. In anembodiment, the processor 204 may be configured to extract the one ormore first content fields of the research paper and the one or moresecond content fields of each of the one or more first resources and theone or more second resources based on the text extraction. Herein, theone or more first content fields and the one or more second contentfields may include or correspond to textual content (for example thetitle, author name, or full description). For example, the processor 204may use a PDF text extraction to extract the one or more first contentfields from a PDF file of the research paper, and extract the one ormore second content fields from a PDF file of the at least one resource(i.e., the one or more first resources and/or the one or more secondresources; or just the first set of resources extracted from the firstwebsite 300A). In another example, the processor 204 may use an OCRtechnique for extraction of the one or more first content fields from ascanned document of the research paper and the one or more secondcontent fields from a scanned document of the at least one resource.

At block 506, the one or more first content fields and the one or moresecond content fields may be extracted based on object detection. In anembodiment, the processor 204 may be configured to extract the one ormore first content fields of the research paper and the one or moresecond content fields of the at least one resource (i.e., the one ormore first resources and/or the one or more second resources; or justthe first set of resources extracted from the first website 300A), basedon the object detection. Herein, the one or more first content fieldsand the one or more second content fields may include (or correspond to)figure or table content (as one or more objects) included in theresearch paper and the extracted resources.

For example, the processor 204 may convert each page (including figuresor tables) in the research paper or a resource into an image. In anexample, the processor 204 may convert a page into a corresponding imageby use of a screen capture or document-to-image conversion technique.The processor 204 may perform object detection on the converted imageusing neural network (such as, deep learning) to detect and extractfigure or table content from the research paper and the extractedresources. Examples of the neural network may include, but are notlimited to, a deep neural network (DNN), a convolutional neural network(CNN), a recurrent neural network (RNN), a CNN-recurrent neural network(CNN-RNN), R-CNN, Fast R-CNN, Faster R-CNN, an artificial neural network(ANN), (You Only Look Once) YOLO network, a Long Short Term Memory(LSTM) network based RNN, CNN+ANN, LSTM+ANN, a gated recurrent unit(GRU)-based RNN, a fully connected neural network, a ConnectionistTemporal Classification (CTC) based RNN, a deep Bayesian neural network,a Generative Adversarial Network (GAN), and/or a combination of suchnetworks. In some embodiments, learning engine associated with theneural network may include numerical computation techniques using dataflow graphs. In certain embodiments, the neural network may be based ona hybrid architecture of multiple Deep Neural Networks (DNNs). In anexample, the processor 204 may use Detectron based neural network todetect and extract the figures. Further, the processor 204 may use atransfer learning on a table dataset such as, but not limited to,TableBank, to detect and extract the tables from the research paper andthe extracted resources (i.e. one or more first resources and the one ormore second resources).

At block 508, the extracted one or more first content fields and the oneor more second content fields may be stored. In an embodiment, theprocessor 204 may be configured to store the extracted one or more firstcontent fields of the research paper and the one or more second contentfields of the at least one resource (i.e., the one or more firstresources and/or the one or more second resources; or just the first setof resources extracted from the first website 300A) in the database 106in a structured format. In an example, the processor 204 may store theone or more first content fields of the research paper and the one ormore second content fields of the first set of resources in the firstdatabase of the database 106. Further, the processor 204 may store theone or more second content fields of the second set of resources in thesecond database of the database 106 and the one or more second contentfields of the third set of resources in the third database of thedatabase 106. Examples of the structured format (which may correspond toa key-value pairs for content fields and corresponding values) mayinclude, but not limited to, a Java Script Object Notation (JSON)format, an eXtensible Markup Language (XML) format, or a Comma SeparatedValues (CSV) format. Control may pass to end.

Although the flowchart 500 is illustrated as discrete operations, suchas 502, 504, 506, and 508. However, in certain embodiments, suchdiscrete operations may be further divided into additional operations,combined into fewer operations, or eliminated, depending on theparticular implementation without detracting from the essence of thedisclosed embodiments.

FIG. 6 illustrates a flowchart of an example method for merger of aplurality of resources associated with a same resource type from one ormore first resources and one or more second resources, arranged inaccordance with at least one embodiment described in the presentdisclosure. FIG. 6 is explained in conjunction with elements from FIG.1, FIG. 2, FIG. 3A, FIG. 3B, FIG. 3C, FIG. 4, and FIG. 5. With referenceto FIG. 6, there is shown a flowchart 600. The method illustrated in theflowchart 600 may start at 602 and may be performed by any suitablesystem, apparatus, or device, such as by the example electronic device102 of FIG. 1 or FIG. 2. Although illustrated with discrete blocks, thesteps and operations associated with one or more of the blocks of theflowchart 600 may be divided into additional blocks, combined into fewerblocks, or eliminated, depending on the particular implementation.

At block 602, a plurality of resources associated with a same resourcetype may be determined from the extracted one or more first resourcesand the one or more second resources. In an embodiment, the processor204 may be configured to determine the plurality of resources (i.e. thatmay be associated with the same resource type) from the one or morefirst resources and the one or more second resources (i.e. extracted atsteps 402 and 408 in FIG. 4). The processor 204 may determine a resourcetype associated with each of the one or more first resources and the oneor more second resources, similar to the identification of the set ofresource types described in step 404 in FIG. 4. For example, withreference to FIG. 3A of the first website 300A and FIG. 3B of the secondwebsite 300B, the processor 204 may determine that a first poster withtitle “Poster-1” (such as, depicted as a link denoted by 304C) and asecond poster with title “Poster-1” (such as, depicted as a link withinthe related resources field of the first publication, denoted by 316A)may be of the same resource type, i.e., a poster resource type. All theresources in the extracted one or more first resources and the one ormore second resources, with same resource type may be considered in thedetermined plurality of resources.

At block 604, the determined plurality of resources may be merged basedon a similarly of content in the determined plurality of resources. Inan embodiment, the processor 204 may be configured to merge thedetermined plurality of resources based on the similarity of content inthe determined plurality of resources. In an embodiment, the pluralityof resources may have a same URL. In another embodiment, the pluralityof resources may have a different URL. In some embodiments, theresources with high similarity of content may have same title orcontent, but may be extracted from different websites or URLs. Theprocessor 204 may check the similarity of content in the determinedplurality of resources based on file comparison. Examples of filecomparison techniques may include, but are not limited to, a bit-wisecomparison, a file or image compression-based comparison, an MPS basedcomparison. Based on the check of similarity of the content between twoor more files of a resource, the processor 204 may merge the two or morefiles and select one of the files, as the file associated with theresource. In an example, the processor 204 may select any of the two ormore files in case of an exact match to merge the files of the resource.In another scenario, the processor 204 may select a file of a largersize or a later date of creation or date of change to merge the files ofthe resource. The processor 204 may merge the resources (i.e. extractedfrom the one or more first websites 112 and the one or more secondwebsites 114) of the same type and/or of the similar content to avoidredundancy or repetition of same resources to be included the final setof resources (i.e. described at steps 410 and 412 in FIG. 4)

At block 606, the first set of resources may be determined based on themerged plurality of resources. In an embodiment, the processor 204 maybe configured to determine the first set of resources based on themerged plurality of resources. For example, with reference to FIGS. 3Aand 3B, the processor 204 may merge a first poster “Poster-1” of thefirst website 300A and a second poster “Poster-1” of the second website300B as a single poster, such as, the first poster, “Poster-1”. In suchcase, the processor 204 may determine the final set of resources as thefirst poster “Poster-1” (denoted by 304C in FIG. 3A) and as the“Presentation Slides-1” (such as, depicted as a link within the relatedresources field of the first publication, denoted by 316A), because the“Poster-1” and the “Presentation Slides-1” correspond to “Title-1” ofthe searched research paper as shown in FIGS. 3A and 3B. Control maypass to end.

Although the flowchart 600 is illustrated as discrete operations, suchas 602, 604, and 606. However, in certain embodiments, such discreteoperations may be further divided into additional operations, combinedinto fewer operations, or eliminated, depending on the particularimplementation without detracting from the essence of the disclosedembodiments.

FIG. 7 illustrates a flowchart of an example method for determination ofa final set of resources associated with a research paper, arranged inaccordance with at least one embodiment described in the presentdisclosure. FIG. 7 is explained in conjunction with elements from FIG.1, FIG. 2, FIG. 3A, FIG. 3B, FIG. 3C, FIG. 4, FIG. 5, and FIG. 6. Withreference to FIG. 7, there is shown a flowchart 700. The methodillustrated in the flowchart 700 may start at 702 and may be performedby any suitable system, apparatus, or device, such as by the exampleelectronic device 102 of FIG. 1 or FIG. 2. Although illustrated withdiscrete blocks, the steps and operations associated with one or more ofthe blocks of the flowchart 700 may be divided into additional blocks,combined into fewer blocks, or eliminated, depending on the particularimplementation.

At block 702, a set of candidate resources and the research paper may bestored. In an embodiment, the processor 204 may be configured to storeeach of the set of candidate resources and the research paper in thedatabase 106, the memory 206, or the persistent data storage 208. Eachof the set of candidate resources and the research paper may include oneor more content fields (for example the one or more first content fieldsand the one or more second content fields described in FIG. 5). Herein,the set of candidate resources may include the media content, forexamples, but is not limited to, a video, a poster, presentation slides,or a program code, which may be associated or related with the researchpaper (i.e. which may of interest for a researcher, such the user 116).In an embodiment, the set of candidate resources and the research papermay be extracted from at least one of the one or more first websites 112or the one or more second websites 114, as described in FIG. 4.

At block 704, the research paper may be parsed into the one or morefirst content fields. In an embodiment, the processor 204 may beconfigured to parse the research paper into the one or more firstcontent fields of the research paper. In an embodiment, the processor204 may parse the research paper into the one or more first contentfields based on at least one of a document layout analysis of a page, adocument or content file, a textual extraction, a document-to-imageconversion, or an object detection as described, for example, in FIGS. 4and 5.

At block 706, each of the one or more first content fields in theresearch paper may be encoded into a first vector. In an embodiment, theprocessor 204 may be configured to encode each of the parsed (orextracted) one or more first content fields in the research paper intothe first vector based on a first field type associated with each of theone or more first content fields of the research paper. The first fieldtype may include, but is not limited to, one of a textual field type, acategorical field type, a date-time field type, a figure field type, ora table field type. In an embodiment, the processor 204 may encode theone or more first content fields based on at least one of a pre-trainedcontextual embedding or a pre-trained bag of embedding for the textualfield type, a categorical encoding for the categorical field type, anumeric encoding for the date-time field type, or a pre-trained encodingfor the figure field type or for the table field type. In an embodiment,the processor 204 may encode a content field into a vector to shortenthe length of the content field to a compressed length such that vectorsof two corresponding content fields of the same lengths and types may becompared to compare the corresponding content fields.

For example, the processor 204 may encode both the title and theabstract (i.e. first content field of the textual field type) of theresearch paper into a vector (such as the first vector) based on a shorttext pre-trained contextual embedding, such as, but not limited to,Bidirectional Encoder Representations from Transformers (BERT). Further,the processor 204 may encode the full text of the research paper into avector (such as the first vector) based on a long text pre-trained bagof embedding, such as, but not limited to, word2vec or fasttext. In anexample, the processor 204 may encode the author name (i.e. firstcontent field of the categorical field type) of the research paper intoa vector (such as the first vector) based on a categorical encoding,such that each author name and/or each institute or organizationassociated with author(s) may be represented as a separate category.Further, the processor 204 may encode the date of publication (i.e.first content field of the date-time field type) of the research paperinto a vector (such as the first vector) based on a numeric encoding.For example, the processor 204 may represent the date of publication asa number that may represent a numeric difference of number ofmilli-seconds between a predefined date-time (such as 12:00 AM on Jan.1, 1970) and 12:00 AM (or the actual time of publication) on the date ofpublication. Further, the processor 204 may encode a figure or table(i.e. first content field of the figured field type or the table fieldtype) in the research paper into a vector (such as the first vector)based on pre-trained encoding, such as, but not limited to, ResidualNetworks (ResNet) pre-trained model or GoogLeNet pre-trained model.

At block 708, each of the stored set of candidate resources may beparsed into the one or more second content fields. In an embodiment, theprocessor 204 may be configured to parse each of the stored set ofcandidate resources into the one or more second content fields. In someembodiments, similar to the parsing of the research paper (as describedin step 704), the processor 204 may perform the parsing of posters andpresentation slides (as the set of candidate resources) to extract theone or more second content fields (for example title, author name, fulltext, date of publications, figures or tables). In certain embodiments,the processor 204 may extract the one or more second content fields of avideo by extraction of a title, a description, author(s) name, and adate-time of the video post. The processor 204 may be further configuredto extract distinct image frames from the video and detect figures andtables in each image frame by object detection based on the deeplearning or other neural network models. The processor 204 may beconfigured to extract the one or more second content fields of a programcode by extraction of the author(s) name, a description, a readme text,and a date-time of the program code upload. The parsing of the one ormore second content fields also described, for example, in FIGS. 4 and5.

At block 710, each of the one or more second content fields in each ofthe parsed set of candidate resources may be encoded into a secondvector. In an embodiment, the processor 204 may be configured to encodeeach of the extracted one or more second content fields (in each of theparsed set of candidate resources) into the second vector based on asecond field type associated with each of the one or more second contentfields. Similar to the first field type, the second field type mayinclude at least one of the textual field type, the categorical fieldtype, the date-time field type, the figure field type, or the tablefield type. In an embodiment, similar to the encoding of the one or morefirst content fields, the processor 204 may encode the one or moresecond content fields based on at least one of the pre-trainedcontextual embedding or a pre-trained bag of embedding for the textualfield type the of second content fields, the categorical encoding forthe categorical field type of the second content fields, the numericencoding for the date-time field type of the second content fields, orthe pre-trained encoding for the figure field type or for the tablefield type of the second content fields. The encoding of the one or moresecond content fields may be similar to the encoding of the one or morefirst content fields based on the corresponding field types for eachcontent field in the set of candidate resources, as described in step706.

At block 712, the encoded one or more first content fields may becompared with the encoded one or more second content fields based on acomparison between the first vector and the second vector. In anembodiment, the processor 204 may be configured to compare the encodedone or more first content fields with the one or more second contentfields based on the comparison between the first vectors of thecorresponding first content fields of the research paper and the secondvectors of the corresponding second content fields of the set ofcandidate resources. In an embodiment, the first vector (related to thefirst content fields of the research paper) may include a set of firsttextual features associated with content fields of the textual fieldtype (e.g., title, abstract, and full text) and a set of firstcategorical features associated with content fields of the categoricalfield type (e.g., authors). The first vector may further include a setof first numeric features associated with content fields of a date-timefield type (e.g., date of publication) and a set of first Siamese neuralnetwork features associated with content fields of a figure field type(e.g., a figure) or table field type (e.g., a table). Similarly, thesecond vector (related to the second content fields of the set ofcandidate resources) may include a set of second textual featuresassociated with content fields of the textual field type and a set ofsecond categorical features associated with content fields of thecategorical field type. The second vector may further include a set ofsecond numeric features associated with content fields of a date-timefield type and a set of second Siamese neural network featuresassociated with content fields of a figure or table field type.

The processor 204 may be configured to compare the first vector (i.e.for each of the encoded one or more first content fields) with thesecond vector for (i.e. for each of the encoded one or more secondcontent fields) based on a comparison of the corresponding sets offeatures. For example, the processor 204 may compare the set of firsttextual features with the set of second textual features, and the set offirst categorical features with the set of second categorical features.Further, the processor 204 may compare the set of first numeric featureswith the set of second numeric features, and the set of first Siameseneural network features with the set of second Siamese neural networkfeatures. The comparison of the first vector with the second vector isdescribed further, for example, in FIG. 8.

At block 714, the final set of resources may be determined based on thecomparison between the first vector for each of the encoded one or morefirst content fields of the research paper, and the corresponding secondvector for each of the encoded one or more second content fields of theset of candidate resources. In an embodiment, the processor 204 may beconfigured to determine the final set of resources based on thecomparison between the first vector and the second vector. Thedetermination of the final set of resources is explained further, forexample, in FIG. 8.

At block 716, a display screen may be controlled to output thedetermined the final set of resources and the research paper. In anembodiment, the processor 204 may be configured to control the displayscreen (such as, the display screen 212 of the electronic device 102) tooutput the determined final set of resources and the research paper. Insome embodiments, the processor 204 may output an integrated UI that maydisplay the final set of resources and the research paper together onthe display screen 212, (for example as the multi-modal output). Anexemplary integrated UI that may display the final set of resources andthe research paper is described further, for example, in FIG. 9. Controlmay pass to end.

Although the flowchart 700 is illustrated as discrete operations, suchas 702, 704, 706, 708, 710, 712, 714, and 716. However, in certainembodiments, such discrete operations may be further divided intoadditional operations, combined into fewer operations, or eliminated,depending on the particular implementation without detracting from theessence of the disclosed embodiments.

FIG. 8 illustrates a flowchart of an example method for determination ofa final set of resources associated with a research paper, arranged inaccordance with at least one embodiment described in the presentdisclosure. FIG. 8 is explained in conjunction with elements from FIG.1, FIG. 2, FIG. 3A, FIG. 3B, FIG. 3C, FIG. 4, FIG. 5, FIG. 6, and FIG.7. With reference to FIG. 8, there is shown a flowchart 800. The methodillustrated in the flowchart 800 may start at 802 and may be performedby any suitable system, apparatus, or device, such as by the exampleelectronic device 102 of FIG. 1 or FIG. 2. Although illustrated withdiscrete blocks, the steps and operations associated with one or more ofthe blocks of the flowchart 800 may be divided into additional blocks,combined into fewer blocks, or eliminated, depending on the particularimplementation.

At block 802, a cosine distance between the set of first textualfeatures associated with the first vector and the set of second textualfeatures associated with the second vector may be calculated. In anembodiment, the processor 204 may be configured to calculate the cosinedistance between the set of first textual features associated with thefirst vector and the set of second textual features associated with thesecond vector for the comparison described, for example, in step 712 inFIG. 7. The cosine distance may be represented by the following equation(1):

$\begin{matrix}{{\cos\mspace{14mu}\theta} = {\frac{A \cdot B}{{A}{B}} = \frac{\sum\limits_{i = 1}^{n}\;{A_{i}B_{i}}}{\sqrt{\sum\limits_{i = 1}^{n}\; A_{i}}\sqrt{\sum\limits_{i = 1}^{n}\; B_{i}}}}} & (1)\end{matrix}$

For example, the value of the cosine distance between the set of firsttextual features (e.g., a feature-set vector “A”) associated with thefirst vector and the set of second textual features (e.g., a feature-setvector “B”) associated with the second vector may lie between “0” and“1”. In an embodiment, a value of cosine distance nearer to ‘1’ or asmall cosine angle of difference (i.e., an angle of difference closer to0°) between the feature-set vector ‘A’ and the feature-set vector ‘B’may indicate a greater similarity between the set of first textualfeatures and the set of second textual features. In an example, theprocessor 204 may determine the similarity between the title of theresearch paper and title of each of the set of candidate resources basedon the calculation of the cosine distance, as described at step 802.

At block 804, an overlap between the set of first categorical featuresassociated with the first vector and the set of second categoricalfeatures associated with the second vector may be calculated. In anembodiment, the processor 204 may be configured to calculate the overlapbetween the set of first categorical features associated with the firstvector and the set of second categorical features associated with thesecond vector for the comparison described, for example, in step 712 inFIG. 7. For example, for the content field of the author (with thecategorical field type), the processor 204 may compare each individualauthor name of the research paper with the author(s) name of the set ofcandidate resources to determine a number of common authors. Theprocessor 204 may calculate the overlap based on a match of at least oneauthor as being common to the research paper and the set of candidateresources. In case two authors are common between the research paper andthe set of candidate resources, the processor 204 may calculate theoverlap as two, and so on.

At block 806, a normalized date-time difference between the set of firstnumeric features associated with the first vector and the set of secondnumeric features associated with the second vector may be calculated. Inan embodiment, the processor 204 may be configured to calculate thenormalized date-time difference between the set of first numericfeatures associated with the first vector and the set of second numericfeatures associated with the second vector for the comparison described,for example, in step 712 in FIG. 7. In an embodiment, the processor 204may be configured to normalize a numeric value of the date-timeassociated with each of the set of first numeric features and the set ofsecond numeric features. For example, the set of first numeric featuresand the set of second numeric features may be numeric representations ofa first date-time (i.e. the first content field in the research paperand a second date-time (i.e. the second content field of the set ofcandidate resources), respectively. The numeric representation of thefirst date-time and the second date-time may be a numeric differencebetween a number of milli-seconds between the predefined date-time (suchas 12:00 AM on Jan. 1, 1970) and the corresponding date-time. Theprocessor 204 may normalize the numeric representation of each of thefirst date-time and the second date-time based on a division of thenumeric representation of each of the first date-time and the seconddate-time with a predefined numeric value. The processor 204 may performa difference between the normalized first date-time and the seconddate-time to calculate the normalized date-time difference between theset of first numeric features associated with the first vector and theset of second numeric features associated with the second vector.

At block 808, a similarity score of a figure or table between the set offirst Siamese neural network features associated with the first vectorand the set of second Siamese neural network features associated withthe second vector may be calculated. In an embodiment, the processor 204may be configured to calculate the similarity score of the figure ortable between the set of first Siamese neural network featuresassociated with the first vector and the set of second Siamese neuralnetwork features associated with the second vector for the comparisondescribed, for example, in step 712 in FIG. 7. In an embodiment, theprocessor 204 may perform a pattern matching based on application of amultiple neural network models (e.g., deep learning) on the set of firstSiamese neural network features and the set of second Siamese neuralnetwork features. Based on the pattern matching, the processor 204 maydetermine the similarity score (e.g., a real number between 0 to 1) as ameasure of similarity between figures or tables in the research paperand a corresponding resource in the set of candidate resources.

At block 810, a first plurality of features associated with the firstvector and a second plurality of features associated with the secondvector may be concatenated to generate a concatenated set of features.In an embodiment, the processor 204 may be configured to concatenate thefirst plurality of features associated with the first vector and thesecond plurality of features associated with the second vector togenerate the concatenated set of features. Herein, each of the firstplurality of features and the second plurality of features may includeat least one of a set of textual features, a set of categoricalfeatures, a set of numeric features, or a set of Siamese neural networkfeatures. For example, the concatenated set of features may include theset of first textual features, the set of second textual features, theset of first categorical features, the set of second categoricalfeatures, the set of first numeric features, the set of second numericfeatures, the set of first Siamese neural network features, and the setof second Siamese neural network features. In an embodiment, theconcatenated set of features may also include values of the calculatedcosine distance (at step 802), the calculated overlap (at step 804), thecalculated normalized date-time difference (at step 806), and thecalculated similarity score (at step 808).

At block 812, it may be determined whether the concatenated set offeatures is sufficient for machine-learning. In an embodiment, theprocessor 204 may be configured to determine whether the concatenatedset of features is sufficient for training a machine-learning model. Inan example, the processor 204 may determine a number of research papersin the database 106 for which the concatenated set of features aregenerated. In case the determined number of research papers is greaterthan a predetermined threshold, the processor 204 may determine that theconcatenated set of features may be sufficient for machine-learning. Insuch case, control may pass to step 814. Otherwise, control may pass tostep 816.

At block 814, a machine-learning model may be generated based ontraining data including a plurality of resources associated with aplurality of research papers. In an embodiment, the processor 204 may beconfigured to generate the machine-learning model based on the trainingdata including the plurality of resources associated with the pluralityof research papers. The plurality of research papers may include theresearch paper, and the plurality of resources may include the final setof resources. For example, the training data may include a data set ofresources with a pre-determined association with corresponding researchpapers in a data set of research papers. The processor 204 may beconfigured to extract the training data from at least one of the one ormore first websites 112 or the one or more second websites 114.

Examples of the machine-learning model may be a classifier or regressionmodel which may be generated based on training to identify arelationship between inputs, such as features in the training data(including research papers and resources) and output labels (such as, amatch or a non-match between a research paper and a resource). Themachine-learning model may be defined by its hyper-parameters, forexample, number of weights, cost function, input size, number of layers,and the like. The hyper-parameters of the machine learning model may betuned and weights may be updated so as to move towards a global minimaof a cost function for the machine learning model. After several epochsof the training on the feature information in the training data, themachine-learning model may be trained to output aprediction/classification result (e.g., a match or non-match output) fora set of inputs (e.g., a research paper and a resource). A predictionresult of the machine-learning model may be indicative of a class label(e.g., a match or non-match output) for each input of the set of inputs(e.g., input features extracted from new/unseen instances of a researchpaper and a resource).

At 816, a rule-based model may be generated based on the concatenatedset of features. In an embodiment, the processor 204 may be configuredto generate the rule-based model based on the concatenated set offeatures. In an example, the processor 204 may generate the rule-basedmodel which may indicate that a resource may be a match with a researchpaper when: the calculated cosine distance for textual features isgreater than “0.5”, the calculated overlap for categorical features isgreater than to “2.0”, the calculated normalized date-time difference isless than “0.1”, and the calculated similarity score is greater than“0.3”.

At 818, at least one of a pre-trained model (e.g., the machine-learningmodel) or the rule-based model may be applied to a first candidateresource from the set of candidate resources to determine whether thefirst candidate resource matches with the research paper. The processor204 may be configured to apply at least one of the pre-trained model(e.g., the machine-learning model) or the rule-based model to the firstcandidate resource from the set of candidate resources. Based on theapplication of one of the pre-trained model or the rule-based model tothe first candidate resource, the processor 204 may determine whetherthe first candidate resource matches with the research paper or not.

At block 820, the final set of resources including the first candidateresource may be determined based on the determination that the firstcandidate resource matches with the research paper. In an embodiment,the processor 204 may determine that the final set of resources mayinclude the first candidate resource based on the determination that thefirst candidate resource matches with the research paper. If the firstcandidate resource matches with the research paper, the processor 204may add the first candidate resource to the final set of resources. Theprocessor 204 may thereby determine the final set of resources that maymatch with the research paper. Thus, the processor 204 of the disclosedelectronic device 102 may automatically extract and match multi-modalonline resources (i.e. presentation slides, video, postures, or programcodes of different resource types) for the research paper of interestfor the user 116 (for example researcher). The processor 204 mayaggregate such extracted and matched multi-modal online resources anddisplay the online resources with the research paper in an integratedUI, as described further, for example, in FIG. 9.

Although the flowchart 800 is illustrated as discrete operations, suchas 802, 804, 806, 808, 810, 812, 814, 816, 818, and 820. However, incertain embodiments, such discrete operations may be further dividedinto additional operations, combined into fewer operations, oreliminated, depending on the particular implementation withoutdetracting from the essence of the disclosed embodiments.

FIG. 9 illustrates an exemplary user interface (UI) that may display afinal set of resources with the research paper, arranged in accordancewith at least one embodiment described in the present disclosure. FIG. 9is explained in conjunction with elements from FIG. 1, FIG. 2, FIG. 3A,FIG. 3B, FIG. 3C, FIG. 4, FIG. 5, FIG. 6, FIG. 7, and FIG. 8. Withreference to FIG. 9, there is shown an integrated UI 900. The integratedUI 900 may include a first research paper 902, a first poster 904, afirst program code 906, a first video 908, and a first presentationslide(s) 910.

For example, as shown in FIG. 9, the first research paper 902 mayinclude a title (e.g., “Title-1”), authors names (e.g., FirstName ALastName A, FirstName B LastName B, FirstName C LastName C, andFirstName D LastName D), affiliation information (e.g., “ABC UniversityUS” associated with FirstName A LastName A and FirstName B LastName B;and “XYZ Inc. US” associated with FirstName C LastName C and FirstName DLastName D), conference volume information (e.g., “Conference C-1, Vol.1”), and date of publication (e.g., “April 2020”). The first researchpaper 902 may further include an abstract (e.g., “Abstract Text-1”) anda full text (e.g., “Full Text-1”). In an embodiment, the integrated UI900 may provide a hyperlink to download the first research paper 902 orto open the first research paper 902 in another user interface.

FIG. 9 further depicts the first poster 904 (e.g., “Poster-1”) that mayinclude infographics related to the first research paper 902. In anexample, the infographics may include a first section includingobjectives of the research, a second section including prior workassociated with the research, a third section including an overview ofthe algorithm associated with the research (e.g., a process-outputgraph), and a fourth section including results of the research as shownin FIG. 9. The integrated UI 900 may provide a link to download thefirst poster 904.

There is further shown in FIG. 9, the first program code 906 (e.g.,“Code-1”) that may depict a representative algorithm or source codeassociated with the first research paper 902, written in a certainprogramming language (e.g., C, C++, Java, C# .NET, Python, or anassembly language). In an embodiment, the integrated UI 900 may includean integrated or embedded debugger or execution environment (such as, aJava Virtual Machine) to debug and/or execute the first program code 906in a real-time and provide an output to the user 116. The integrated UI900 may further provide an interface to the user 116 to edit the firstprogram code 906 in real-time and export the first program code 906 toanother Integrated Development Environment (IDE). In an embodiment, theintegrated UI 900 may include a link to a source code repository thatmay host the first program code 906. In another embodiment, theintegrated UI 900 may embed a web page of the source code repositoryassociated with the first program code 906.

There is further shown in FIG. 9, the first video 908 (e.g., “Video-1”)that may depict multi-media (e.g., audio/video) content associated withthe first research paper 902. The integrated UI 900 may include a videoplayer to play-back the first video 908 based on a user input from theuser 116. The first video 908 may represent the content of the firstresearch paper 902 and/or may include a commentary (including closedcaptioned texts) for the description of the concepts related to theresearch paper. The integrated UI 900 may provide a link to download thefirst video 908.

There is further shown in FIG. 9, the first presentation slides 910(e.g., “Presentation Slides-1”) that may depict multi-media content(e.g., a presentation slide deck with graphics, animation, voice-over,commentary, and/or text) associated with the first research paper 902.The integrated UI 900 may include a slide player to present the firstpresentation slides 910 to the user 116. The integrated UI 900 mayprovide a link to download the first presentation slides 910.

It should be noted that the integrated UI 900, the first research paper902, the first poster 904, the first program code 906, the first video908, and the first presentation slide 910 shown in FIG. 9 are presentedmerely as examples and should not be construed to limit the scope of thedisclosure.

Typically, a researcher may understand the state of art of a domain ofinterest by study of multiple research papers of the domain of interest.However, this may be tedious to analyze every such research paper.Hence, the researcher may search for online resources associated witheach research paper to get a quick overview of the research papers orthe related domain. The researcher may have to manually search andextract each online resource related to the research papers frommultiple websites or search engines that may be scattered across theInternet. Further, the researcher may have to manually compare contentavailable from each online resource, with the research papers toestablish a correlation between the online resources and the researchpapers to further determine whether an extracted online resourceactually corresponds to a certain research paper of interest or not. Asmay be evident, the manual process of extraction of the online resourcesassociated with a research paper may be time consuming and may not scalewell to a batch of a large number of research papers. Therefore, thedisclosed electronic device 102 may provide automated extraction ofmulti-modal online resources including the media content (poster, video,slides, or codes) which may be associated with a research paper, and mayfurther provide automated validation that the extracted online resourcescorrespond to (or match with) the research paper, in comparison toconventional solutions of manual extraction and validation of the onlineresources. Further, the disclosed electronic device 102 may provide anintegrated UI (e.g., the integrated UI 900 in FIG. 9) for an integratedor user friendly display of the multi-modal online resources (e.g.,posters, videos, presentation slides, and program codes) along with theresearch paper associated with the multi-modal online resources. Suchintegrated UI, as shown for example, in FIG. 9 may provide adashboard-based interface to the researcher (i.e. user 116) to study theonline resources (i.e. matched with a target research paper) togetherwith the research paper and efficiently organize the displayed onlineresources on the integrated UI. The automatic extraction, validation,and the integrated UI provided by the disclosed electronic device 102may further provide a better and enhanced understanding of the researchpaper and the domain of interest associated with the research paper,while saving substantial time for the user 116 (i.e. researcher or anyknowledge worker) to search and collate such resources.

Various embodiments of the disclosure may provide one or morenon-transitory computer-readable storage media configured to storeinstructions that, in response to being executed, cause a system (suchas the example electronic device 102) to perform operations. Theoperations may include extracting, from one or more first websites, oneor more first resources based on a title associated with a researchpaper. The operations may further include identifying a set of resourcetypes associated with the extracted one or more first resources. Theoperations may further include determining one or more first resourcetypes from a predefined plurality of resource types based on theidentified set of resource types. The set of resource types may excludethe determined one or more first resource types. The operations mayfurther include extracting, from one or more second websites, one ormore second resources, associated with the determined one or more firstresource types, based on the title associated with the research paper.Each of the one or more first resources and the one or more secondresources comprises media content. The operations may further includedetermining a final set of resources based on a comparison between oneor more first content fields of the research paper and one or moresecond content fields of the extracted one or more first resources andthe extracted one or more second resources. The operations may furtherinclude controlling a display screen to output the determined final setof resources and the research paper.

Various other embodiments of the disclosure may provide one or morenon-transitory computer-readable storage media configured to storeinstructions that, in response to being executed, cause a system (suchas the example electronic device 102) to perform operations. Theoperations may include storing a set of candidate resources and aresearch paper, each including one or more content fields. The set ofcandidate resources may include media content and the set of candidateresources may be associated with the research paper. The operations mayfurther include encoding each of one or more first content fields in theresearch paper into a first vector based on a first field typeassociated with each of the one or more first content fields. Theoperations may further include parsing each of the stored set ofcandidate resources into one or more second content fields. Theoperations may further include encoding each of the one or more secondcontent fields in each of the parsed set of candidate resources into asecond vector, based on a second field type associated with each of theone or more second content fields. The operations may further includecomparing the first vector for each of the encoded one or more firstcontent fields with the second vector for each of the encoded one ormore second content fields. The operations may further includedetermining a final set of resources based on the comparison. Theoperations may further include controlling a display screen to outputthe determined final set of resources and the research paper.

As used in the present disclosure, the terms “module” or “component” mayrefer to specific hardware implementations configured to perform theactions of the module or component and/or software objects or softwareroutines that may be stored on and/or executed by general purposehardware (e.g., computer-readable media, processing devices, etc.) ofthe computing system. In some embodiments, the different components,modules, engines, and services described in the present disclosure maybe implemented as objects or processes that execute on the computingsystem (e.g., as separate threads). While some of the system and methodsdescribed in the present disclosure are generally described as beingimplemented in software (stored on and/or executed by general purposehardware), specific hardware implementations or a combination ofsoftware and specific hardware implementations are also possible andcontemplated. In this description, a “computing entity” may be anycomputing system as previously defined in the present disclosure, or anymodule or combination of modulates running on a computing system.

Terms used in the present disclosure and especially in the appendedclaims (e.g., bodies of the appended claims) are generally intended as“open” terms (e.g., the term “including” should be interpreted as“including, but not limited to,” the term “having” should be interpretedas “having at least,” the term “includes” should be interpreted as“includes, but is not limited to,” etc.).

Additionally, if a specific number of an introduced claim recitation isintended, such an intent will be explicitly recited in the claim, and inthe absence of such recitation no such intent is present. For example,as an aid to understanding, the following appended claims may containusage of the introductory phrases “at least one” and “one or more” tointroduce claim recitations. However, the use of such phrases should notbe construed to imply that the introduction of a claim recitation by theindefinite articles “a” or “an” limits any particular claim containingsuch introduced claim recitation to embodiments containing only one suchrecitation, even when the same claim includes the introductory phrases“one or more” or “at least one” and indefinite articles such as “a” or“an” (e.g., “a” and/or “an” should be interpreted to mean “at least one”or “one or more”); the same holds true for the use of definite articlesused to introduce claim recitations.

In addition, even if a specific number of an introduced claim recitationis explicitly recited, those skilled in the art will recognize that suchrecitation should be interpreted to mean at least the recited number(e.g., the bare recitation of “two recitations,” without othermodifiers, means at least two recitations, or two or more recitations).Furthermore, in those instances where a convention analogous to “atleast one of A, B, and C, etc.” or “one or more of A, B, and C, etc.” isused, in general such a construction is intended to include A alone, Balone, C alone, A and B together, A and C together, B and C together, orA, B, and C together, etc.

Further, any disjunctive word or phrase presenting two or morealternative terms, whether in the description, claims, or drawings,should be understood to contemplate the possibilities of including oneof the terms, either of the terms, or both terms. For example, thephrase “A or B” should be understood to include the possibilities of “A”or “B” or “A and B.”

All examples and conditional language recited in the present disclosureare intended for pedagogical objects to aid the reader in understandingthe present disclosure and the concepts contributed by the inventor tofurthering the art, and are to be construed as being without limitationto such specifically recited examples and conditions. Althoughembodiments of the present disclosure have been described in detail,various changes, substitutions, and alterations could be made heretowithout departing from the spirit and scope of the present disclosure.

1. A method, comprising: extracting, from one or more first websites,one or more first resources based on a title associated with a researchpaper; identifying a set of resource types associated with the extractedone or more first resources; determining one or more first resourcetypes from a predefined plurality of resource types based on theidentified set of resource types, the set of resource types exclude thedetermined one or more first resource types, the predefined plurality ofresource types comprises at least one of: a research paper type, aposter type, a presentation slide type, a video type, or a program codetype; extracting, from one or more second websites, one or more secondresources, associated with the determined one or more first resourcetypes, based on the title associated with the research paper, each ofthe one or more first resources and the one or more second resourcescomprises media content; determining a final set of resources based on acomparison between one or more first content fields of the researchpaper and one or more second content fields of the extracted one or morefirst resources and the extracted one or more second resources, thedetermining being based on when the comparison results in the one ormore first content fields matching the one or more second contentfields; and controlling a display screen to output the determined finalset of resources and the research paper, the method enabling a betterand enhanced understanding of the research paper and a domain ofinterest associated with the research paper based at least in part onthe predefined plurality of resources types.
 2. (canceled)
 3. The methodaccording to claim 1, wherein the one or more first websites comprise atleast one of: a first website associated with a conference or journal ofresearch papers, a second website associated with an author or publisherof research papers, or a third website associated with a resource type.4. The method according to claim 1, further comprising extracting one ormore first meta fields from the research paper and one or more secondmeta fields from each of the one or more first resources and the one ormore second resources.
 5. The method according to claim 1, furthercomprising extracting the one or more first content fields and the oneor more second content fields based on text extraction, wherein each ofthe one or more first content fields and the one or more second contentfields comprises textual content.
 6. The method according to claim 1,further comprising: extracting the one or more first content fields andthe one or more second content fields based on object detection, whereineach of the one or more first content fields and the one or more secondcontent fields corresponds to at least one of figure content or tabularcontent as an object.
 7. The method according to claim 1, furthercomprising: determining a plurality of resources associated with a sameresource type from the extracted one or more first resources and theextracted one or more second resources; merging the determined pluralityof resources, based on a similarity of content in the determinedplurality of resources; and determining the final set of resources basedon the merged plurality of resources.
 8. A method, comprising: storing aset of candidate resources and a research paper, each including one ormore content fields, the set of candidate resources comprise mediacontent and are associated with the research paper; encoding each of oneor more first content fields in the research paper into a first vectorbased on a first field type associated with each of the one or morefirst content fields; parsing each of the stored set of candidateresources into one or more second content fields; encoding each of theone or more second content fields in each of the parsed set of candidateresources into a second vector, based on a second field type associatedwith each of the one or more second content fields; comparing the firstvector for each of the encoded one or more first content fields with thesecond vector for each of the encoded one or more second content fields;determining a final set of resources based on the comparison, thedetermining being based on when the comparison results in the firstvector matching the second vector; generating a machine-learning modelbased on training data including a plurality of resources associatedwith a plurality of research papers, the plurality of resourcescomprising at least one of: a research paper type, a poster type, apresentation slide type, a video type, or a program code type, theplurality of research papers include the research paper and theplurality of resources include the final set of resources; andcontrolling a display screen to output the determined final set ofresources and the research paper, the method enabling a better andenhanced understanding of the research paper and a domain of interestassociated with the research paper based at least in part on thepredefined plurality of resources types.
 9. The method according toclaim 8, further comprising parsing the research paper into the one ormore first content fields based on at least one of: a document layoutanalysis, a textual extraction, a document-to-image conversion, or anobject detection.
 10. The method according to claim 8, wherein each ofthe first field type and the second field type comprises one of: atextual field type, a categorical field type, a date-time field type, afigure field type, or a table field type.
 11. The method according toclaim 10, wherein each of the one or more first content fields and theone or more second content fields is encoded based on at least one of: apre-trained contextual embedding or a pre-trained bag of embedding forthe textual field type, a categorical encoding for the categorical fieldtype, a numeric encoding for the date-time field type, or a pre-trainedencoding for the figure field type or for the table field type.
 12. Themethod according to claim 8, wherein when each of the first field typeand the second field type is a textual field type, the comparisonfurther comprising calculating a cosine distance between a set of firsttextual features associated with the first vector and a set of secondtextual features associated with the second vector.
 13. The methodaccording to claim 8, wherein when each of the first field type and thesecond field type is a categorical field type, the comparison furthercomprising calculating an overlap between a set of first categoricalfeatures associated with the first vector and a set of secondcategorical features associated with the second vector.
 14. The methodaccording to claim 8, wherein when each of the first field type and thesecond field type is a date-time field type, the comparison furthercomprising calculating a normalized date-time difference between a setof first numeric features associated with the first vector and a set ofsecond numeric features associated with the second vector.
 15. Themethod according to claim 8, wherein when each of the first field typeand the second field type is a figure field type or a table field type,the comparison further comprising calculating a similarity score of afigure or a table between a set of first Siamese neural network featuresassociated with the first vector and a set of second Siamese neuralnetwork features associated with the second vector.
 16. The methodaccording to claim 8, further comprising concatenating a first pluralityof features associated with the first vector and a second plurality offeatures associated with the second vector to generate a concatenatedset of features, wherein each of the first plurality of features and thesecond plurality of features comprises at least one of: a set of textualfeatures, a set of categorical features, a set of numeric features, or aset of Siamese neural network features.
 17. (canceled)
 18. The methodaccording to claim 16, further comprising: generating a rule-based modelbased on the concatenated set of features; and determining the final setof resources based on the generated rule-based model.
 19. The methodaccording to claim 16, further comprising: applying at least one of: amachine-learning model or a rule-based model to a first candidateresource from the set of candidate resources to determine whether thefirst candidate resource matches with the research paper; anddetermining the final set of resources including the first candidateresource based on the determination that the first candidate resourcematches with the research paper.
 20. One or more non-transitorycomputer-readable storage media configured to store instructions that,in response to being executed, cause a system to perform operations, theoperations comprising: extracting, from one or more first websites, oneor more first resources based on a title associated with a researchpaper; identifying a set of resource types associated with the extractedone or more first resources; determining one or more first resourcetypes from a predefined plurality of resource types based on theidentified set of resource types, the set of resource types excludes thedetermined one or more first resource types, the predefined plurality ofresource types comprises at least one of: a research paper type, aposter type, a presentation slide type, a video type, or a program codetype; extracting, from one or more second websites, one or more secondresources, associated with the determined one or more first resourcetypes, based on the title associated with the research paper, whereineach of the one or more first resources and the one or more secondresources comprises media content; determining a final set of resourcesbased on a comparison between one or more first content fields of theresearch paper and one or more second content fields of the extractedone or more first resources and the extracted one or more secondresources, the determining being based on when the comparison results inthe one or more first content fields matching the one or more secondcontent fields; and controlling a display screen to output thedetermined final set of resources and the research paper, the methodenabling a better and enhanced understanding of the research paper and adomain of interest associated with the research paper based at least inpart on the predefined plurality of resources types.
 21. One or morenon-transitory computer-readable storage media configured to storeinstructions that, in response to being executed, cause a system toperform operations, the operations comprising: storing a set ofcandidate resources and a research paper, each including one or morecontent fields, the set of candidate resources comprise media contentand are associated with the research paper; encoding each of one or morefirst content fields in the research paper into a first vector based ona first field type associated with each of the one or more first contentfields; parsing each of the stored set of candidate resources into oneor more second content fields; encoding each of the one or more secondcontent fields in each of the parsed set of candidate resources into asecond vector, based on a second field type associated with each of theone or more second content fields; comparing the first vector for eachof the encoded one or more first content fields with the second vectorfor each of the encoded one or more second content fields, thedetermining being based on when the comparison results in the firstvector matching the second vector; determining a final set of resourcesbased on the comparison; generating a machine-learning model based ontraining data including a plurality of resources associated with aplurality of research papers, the plurality of resources comprising atleast one of: a research paper type, a poster type, a presentation slidetype, a video type, or a program code type, the plurality of researchpapers include the research paper and the plurality of resources includethe final set of resources; and controlling a display screen to outputthe determined final set of resources and the research paper, the methodenabling a better and enhanced understanding of the research paper and adomain of interest associated with the research paper based at least inpart on the predefined plurality of resources types.